Online Adjusting Task Models for Ubiquitous Robotic Systems

Task modeling and task planning are very important in robotic systems especially for large-scale nondeterministic problems. Two widely studied models (the classical planning model and the Markov Decision Process (MDP) model) are inapplicable to such problems due to either inherently assumed determinism or dimensional explosion. An amalgamation of these two results in a new model which is proposed in this study under the name “Reduced Markov Decision Process” (RMDP) model. This new model simplifies the conventional MDP model by reducing the branching factor of state transitions. Further, based on the RMDP model, a modified Dynamic Programming (DP) algorithm is proposed. The RMDP model also facilitates online learning that adapts the model to environmental changes. A “forgetting” model is employed for this online adjustment. In the experiment, a ubiquitous robotic system is implemented for robotic bar-tending task. The results demonstrate that the model conveniently facilitates online-updating to better match the real environment.